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New York City’s Charter Schools 


Executive Summary 


his report evaluates the current state of research on New York City charter schools. 
Overall, their effect on student performance is unambiguously positive. But the 
research is more dated and limited in scope than proponents and critics of charters 


appreciate. 


While the research on charter schools across the U.S. is growing, this paper focuses exclusively on studies that 
evaluate one or more aspects of New York City charter schools. This focus is important because the characteris- 
tics and effects of charter schools vary from city to city and, indeed, from charter to charter. 


Key Findings 


Y 


Y 


Students who attend a New York City charter instead of a traditional public school do much better on math tests 
and somewhat better on English language arts (ELA) tests. These positive effects appear to have remained similar 
over time, even as the number of charters and students has expanded. However, the studies that employ the 
strongest potential research design are dated or targeted only at a few highly effective schools that do not 
represent the full charter sector today. 


New York City charter schools are not equally effective. About half appear to be more effective than the traditional 
public school that students would have otherwise attended. A small percentage of charters appear to have 
negative effects on student test scores. 


New York City charters operated by a charter management organization appear to be more effective than other 
charters, on average. School attributes—such as frequent teacher feedback, data-guided instruction, tutoring, 
high standards for students, and additional instruction time—are related to larger charter school effects. Variations 
in class size and the number of teachers with advanced degrees are not related to the effectiveness 

of charter schools. 


Differences in resources do not explain differences in effectiveness between charter schools and traditional public 
schools or between charters in New York City. To date, charter schools have received fewer public resources 
than have traditional public schools, but recent changes in how charters are funded might narrow the gap. 

There is large variation in the resources that charter schools receive from nonpublic sources. However, differences 
in overall per-pupil funding do not appear to be associated with school quality. 


There is no evidence that New York City charter schools systematically push out low-performing students. 
In fact, low-performing students are less likely to exit charter schools than they are to exit traditional public 
schools, especially after accounting for differences in their demographic characteristics. 


Competition from New York City charter schools has either no effect, or a positive effect, on the performance 
of students in the nearby traditional public schools. 


NEW YORK CITY'S 
CHARTER SCHOOLS 


What the Research Shows 


Introduction 


f you ask five people what the academic research says about New York 

City charter schools, you might get seven different answers. Indeed, the 

empirical evidence is now so broad and long-standing that it is worth 
stepping back to reflect on where it stands. 


This report reviews existing research, which shows that New York City charter schools have an un- 
ambiguously positive effect on student performance. But the research is more dated and limited 
in scope than is commonly appreciated. 


I focus exclusively on the results from studies that specifically evaluate one or more aspects of 
New York City’s charter sector because the characteristics and impacts of charter schools vary 
dramatically by locality.t In New York and elsewhere, conversations about the impact of charter 
schools are often contaminated by discussions of the national charter school literature. That lit- 
erature provides a broad understanding of the charter school movement and offers insights into 
why charters appear to be more effective in some localities than in others. But in cases like New 
York City, where broadly informative local research is available, the policy conversation should 
focus explicitly on the local body of research. (In cases where research on non—New York charter 
school sectors might inform this report, I include a brief description in the endnotes.) 


I have done my best to collect all of the relevant academic literature on New York City charter 
schools in this review. (See the Appendix for further readings.) I do not impose a firm cutoff date 
to include a paper, but I do exclude early studies that are clearly no longer relevant. In addition 
to employing my own knowledge of the literature, I searched on Google Scholar, browsed the 
websites of organizations—supporters as well as opponents—that frequently write about the city’s 
charter sector in the popular press,? and collected further relevant papers that were cited in the 
initial papers that I identified. 


I do not restrict the review to published academic work. Peer review is an imperfect signal of a 
study’s quality. In some cases, I reference published critiques of a particular study, but I do not 
treat such reviews as evaluations. I organize this review as a series of questions and answers—a 
Q&A format seemed best for addressing the most pressing claims and controversies among the 
public, as well as among the scholarly community. 


New York City’s Charter Schools 


What Is the Effect of 
Attending a Charter 
School on Student 
Achievement? 


Large and positive. Attending a charter school instead of 
a traditional public school has a large, positive effect on 
student achievement in math and a smaller positive effect 
in ELA. The effect of attending a charter school appears 


to have remained similar over time, even as the sector 
has expanded dramatically. Unfortunately, studies using 
the strongest potential research design are dated or 
targeted at only a few highly effective schools that do not 
represent the full charter sector today. 


New York City charter schools post substantially 
higher test scores, on average, than do the city’s tradi- 
tional public schools. Alas, the aggregate test scores of 
charter and traditional public schools are not directly 
comparable. Differences in the performance of charter 
and traditional public school students might reflect dif- 
ferences in the quality of schooling, student differenc- 
es, or both. Measuring the effect of attending a charter 
school on student performance requires rigorous sta- 
tistical analysis. 


Seven studies estimate the causal effect of attending 
a New York City charter school on student perfor- 
mance—i.e., they attempt to directly answer the ques- 
tion of whether the same charter school student would 
have performed differently had he or she attended a 
traditional public school.‘ These studies use one of two 
research designs, both of which have benefits and lim- 
itations that should be considered when interpreting 
their results. 


Five use a randomized field trial (RFT) design. This 
design takes advantage of the fact that oversubscribed 
charter schools are required to use a random lottery to 
offer students the opportunity to enroll. The research- 
ers compare the later outcomes of students who were 
randomly offered a charter school seat with those of 
students who also applied but were randomly denied 
the opportunity to enroll. 


Because winning the charter lottery is not related to 
any other student characteristic, any differences in the 
later outcomes of lottery winners and losers are either 
random or are directly due to the opportunity to enroll 
in acharter school. Researchers then statistically adjust 
the estimated effect of being offered a charter seat 
(the “reduced form” or “intention-to-treat” estimate) 
to account for the fact that not all students who are 
offered a charter school seat actually attend the school 


(the treatment-on-the-treated estimate). Because the 
RFT design accounts for all pretreatment differences 
between the treatment and control groups, it is widely 
considered to be the gold standard for research designs 
in the social sciences. 


From a policy perspective, RFT studies produce limited 
information because their results strictly apply only to 
those represented in the lottery pools. Thus, if students 
who apply to charter schools are systematically differ- 
ent from those who do not apply to them, then the es- 
timates from an RFT study likely would not apply to 
non-applicants.5 Many RFT studies of New York City 
charter schools also focus exclusively on one or several 
charter operators; and the schools included are typical- 
ly among the highest-performing. Such studies reveal 
a great deal about the effect of attending a particular 
charter school, but they do not offer a complete picture 
of the effectiveness of New York’s charter school sector. 


The second research design, “matching,” uses a com- 
puter algorithm to compare the performance of each 
student attending a charter school with a similar 
student or students attending a traditional public 
school. The Center for Research on Educational Out- 
comes (CREDO) at Stanford University has used this 
design to estimate the effects of charter schools on 
student performance in New York City and elsewhere 
(two of its matching studies are discussed below). 
Unlike the RFT design, matching allows the research- 
er to include all charter schools, regardless of whether 
they are oversubscribed or participate in the evalua- 
tion. 


Matching requires stronger assumptions than RFT to 
produce estimates of the causal impact of attending 
a charter school. RFT studies control for differences 
between applicants who are offered and not offered a 
charter seat, whether or not the difference is observed 
in the researcher’s data set. Matching studies match 
students only according to characteristics that are ob- 
served in the data set, even though students attending 
charters may differ from those attending traditional 
public schools in unobserved ways, such as the level of 
parental involvement. 


Figures 1—2 plot the main estimates for the impact of 
attending a New York City charter school on student 
math and ELA scores, as reported in the seven studies.° 
The five RFT studies (blue dots) list the school includ- 
ed in the analysis, as well as the studies’ authors. The 
two matching studies (yellow dots) are CREDO (2013) 
and CREDO (2017); their estimates hold for all charter 
schools operating in a given year in New York City. 


Figures 1—2 show that the estimated effect of attending 
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“Indicates whether the result is statistically significant at the 95% confidence level.’ The x-axis represents the final observed year in the study's sample. The y-axis is the magnitude of the 
study’s main estimate (i.e., the effect of a year of charter schooling, measured in standard deviation units).* For example, to obtain the study’s estimate for the effect of four years of charter 
schooling, one would multiply the study's estimate by 4.° For comparison, the most cited study, Krueger (1999), evaluating the impact of small class sizes, found that students in small classes 


outperformed students in regular-size classes by about 0.22 standard deviations after four years, which—when translated into a constantly accumulating single-year effect—would be an 
average of 0.055 standard deviations.'° 


Source: Author’s compilation of seven studies cited in endnote 4 
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“Indicates whether the result is statistically significant at the 95% confidence level.’’ The x-axis represents the final observed year in the study’s sample. The y-axis is the magnitude of the 
study’s main estimate (i.e., the effect of a year of charter schooling, measured in standard deviation units).'* For example, to obtain the study's estimate for the effect of four years of charter 
schooling, one would multiply the study's estimate by 4.'° For comparison, the most cited study, Krueger (1999), evaluating the impact of small class sizes, found that students in small classes 


outperformed students in regular-size classes by about 0.22 standard deviations after four years, which—when translated into a constantly accumulating single-year effect—would be an 
average of 0.055 standard deviations." 


Source: Author’s compilation of seven studies cited in endnote 4 
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FIGURE 3 
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a New York City charter school is positive in all but 
one case; 23 of 28 estimates are significantly posi- 
tive. Attending a New York City charter appears to 
have a larger effect in math than in ELA. The mag- 
nitude of the positive effect in math is substantial, 
especially given that the estimate covers only a year 
of charter school attendance. The smallest esti- 
mate suggests that four years of attending a charter 
school would lead to a 0.28 standard deviation in- 
crease in student math scores (0.07 x 4 years). The 
estimates in math are also quite stable over time 
and across studies. 


Meanwhile, the estimated effects on student perfor- 
mance in ELA are more modest and less consistent. 
However, the evidence suggests a significant positive 
effect in ELA that would be meaningful after accu- 
mulating over time. 


As noted, RFT and matching studies have strengths 
and weaknesses. Three of the RFT studies evalu- 
ate only a single, high-performing charter network 
(Promise Academy, Democracy Prep, or Success 
Academy). A fourth uses information from a more 
expansive group of 29 schools. Hoxby and Murarka 
(2009), the fifth RFT study and the only oneto include 
all of the city’s then-operating charter schools, found 
a significant positive impact in math and ELA. 


Though Hoxby and Murarka remain highly influen- 
tial in the policy discussion, their results are quite 
dated. New York City’s charter sector has grown 
dramatically (Figure 3) since 2006, the final year 
of study in Hoxby and Murarka. In 2006, only 47 
charter schools operated in New York, compared 
with 227 today. Because of this big expansion, the 
effect of attending a charter school may have changed 
over time. 


2012 2013 2014 2015 2016 2017 2018 


If, say, the city successfully closed ineffective char- 
ters and encouraged the opening of only highly ef- 
fective ones, the impact of attending a charter school 
would tend to increase over time. On the other hand, 
the quality of charter schools might diminish as the 
sector expands and digs deeper into the labor pool 
for administrators and teachers, or if the addition- 
al students who enroll are, for some reason, less 
responsive to charter schooling than the average 
student who enrolled previously. 


Compare matching and RFT still further. The fact 
that the RFT studies often include only a small mi- 
nority of charter schools likely explains why they 
tend to produce substantially larger effect size. 
Meanwhile, each matching estimate produces a sta- 
tistically significant result; but four of 10 RFT es- 
timates are statistically insignificant. The reason: 
the RFT estimates are measured imprecisely, partly 
because they utilize far fewer schools and students 
than do the matching estimates. 


Are the matching estimates reasonable? As of 2006, 
the estimated impact of attending a charter school 
using RFT (0.06 standard deviations in ELA and 0.09 
standard deviations in math) was similar to the esti- 
mated impact of attending a charter school in 2007 
using matching (0.10 standard deviations in ELA and 
0.12 standard deviations in math). The fact that these 
approaches—applied at about the same time, using 
nearly the same sample of schools—produced relative- 
ly similar results suggests that matching is capable of 
producing causal estimates for the effect of attending 
a New York City charter school. Indeed, the matching 
estimates have been relatively consistent over time, es- 
pecially in math. This suggests that the average effec- 
tiveness of the city’s charter sector has not diminished, 
even as it has expanded." 


Are All Charter Schools 
Equally Effective? 


No. There is wide variation in the effectiveness of 
charter schools. About half appear to be more effective 


than the traditional public schools that students would 
otherwise have attended. A small percentage appear to 
have negative effects on student test scores. 


The empirical research described above strongly sug- 
gests that, on average, students attending a charter 
school score higher in math and ELA than they would 
have had they attended a traditional public school. 
However, charters are not monolithic: they are sep- 
arate, independently operated, schools. The type of 
schooling offered by charters varies substantially in 
New York, too. Charters also have access to varying 
levels of financial resources, from public and private 
sources. Thus, while charter schools might be effective 
on average, not all New York City charter schools are 
equally effective. 


The potential for variation in the effectiveness of 
charter schools is visible in Figures 1-2. As discussed, 
the RFT studies tend to produce larger estimated 
effects than do matching studies. Some of that differ- 
ence could result from differences in research meth- 
odologies. But it could also be due to the fact that the 
RFT studies focus on a set of highly effective schools, 
which, by definition, implies variation in charter school 
quality. Dobbie and Fryer (2013) is the only RFT study 
to produce an estimated charter school effect (in math) 
worse than those produced by matching estimates in 
a given year. It is also the only RFT study since Hoxby 
and Murarka (2009) to include a broad set of charter 
schools rather than only those in a single network. 


The matching studies offer direct evidence that the 
quality of New York’s charters varies, too. CREDO 
(2017), which disaggregates results by school, finds 
that 47.7% of the city’s charter schools produced sig- 
nificant gains in math when compared with tradition- 
al public schools, 35.3% scored equally well, and 17% 
scored worse. It finds similar results for ELA. 


CREDO (2013) and CREDO (2017) also find variation 
in the quality of charter school networks. Among net- 
works operating at least four charters in New York, the 
effect of attending a network school ranges from —0.06 
to 0.24 standard deviations in ELA and (an insignifi- 
cant) 0.00 to 0.42 standard deviations in math. 


What School-Based 
Factors Explain a Charter 
School’s Effectiveness 
Relative to Other Charter 
Schools? 


Charter management organizations, frequent teach- 
er feedback, data-guided instruction, tutoring, high 
standards for students, and additional instruction time 


all appear to contribute significantly to a charter’s 
effectiveness. But variations in school inputs, including 
class size and teachers with advanced degrees, do not 
appear to be related to effectiveness. 


Some studies attempt to identify characteristics of 
successful charter schools by measuring each school’s 
impact on student performance, using RFT or match- 
ing, and then they correlate the measured effect of 
each school with one or more characteristics of the 
school. Keep in mind that the estimated charter school 
effect can be considered to be causal, but the associ- 
ations between effectiveness and observed character- 
istics of the school cannot be considered to be causal. 
(The attributes of these schools are largely fixed and 
are not randomly assigned, which makes it impossible 
to distinguish between their effects and the effects of 
the schools’ unobserved attributes.) These studies can 
therefore identify characteristics that tend to be found 
in the more successful schools, but they cannot prove 
that the same (positive) results would occur if adopted 
in other schools. 


CREDO (2017) finds that charter schools operated by 
charter management organizations (CMOs)—organi- 
zations that manage networks of several schools under 
a common leadership and philosophy—tend to have 
substantially larger positive effects on their students 
than do charters operated by non-CMOs. (For charter 
schools operated by non-CMOs, the study still finds a 
significantly positive effect in math but an insignificant 
effect in ELA.) 


The RFT studies that utilize several different charter 
schools also attempt to find correlations between char- 
acteristics and measured school quality. Hoxby and 
Murarka (2009), for example, find that longer school 
years are associated with superior quality. But they do 
not find other characteristics that are robustly associ- 
ated with measured school effectiveness. 


Dobbie and Fryer (2013) present the most thorough 
analysis of the underlying factors associated with 
charter school quality to date. They supplement report- 
ed information for each school with detailed survey 
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data and, in some cases, video surveillance of the 
school’s operations. They found that an index of five 
characteristics—frequent teacher feedback, data-guid- 
ed instruction, tutoring, high standards for students, 
and additional instruction time—explained nearly half 
the variation in measured charter school quality.'® 
These characteristics are often found in “no excuses” 
charter schools.” 


Meanwhile, Dobbie and Fryer (2013) find no signif- 
icant correlation between measured charter school 
effects and inputs, such as whether the pupil-teacher 
ratio is less than 13 to 1, whether per-pupil spending 
is greater than $15,000, or whether more than 11% 
of teachers have an advanced degree. The one school 
input found to be significantly related to charter school 
effectiveness: schools where more than 89% of teach- 
ers are uncertified perform significantly worse in math 
and ELA. 


Do Differences in 
Resources Explain 
Charter School Effects 
Relative to Traditional 
Public Schools and Other 
Charters? 


No. Differences in per-pupil funding do not appear to be 
associated with school quality. To date, charter schools 


have received fewer resources than traditional public 
schools, though there is large variation in the resources 
that charter schools receive from nonpublic sources. 


Differences in financial resources could influ- 
ence the effectiveness of charters, relative to tra- 
ditional public schools and other charters. Here, 
too, the empirical research may be dated. In 2011, 
the tuition formula for state funding to charter 
schools was frozen, with some allowances for sup- 
plemental aid in later years. But in April 2017, 
New York State increased charter school funding 
by $500 per pupil, tied future charter school 
funding to changes in traditional public school 
spending, and increased the amount of capital 
funding for charters not located in a traditional 
public school building.’® As a result, previous com- 
parisons between the funding of charter and tradi- 
tional public schools do not necessarily reflect the 
current state of charter school finance in New York 
City (Belfield 2017). 


Nevertheless, some aspects of the academic liter- 
ature remain pertinent. In particular, the gains 
produced by charter schools—relative to the city’s 
traditional public schools and to one another— 
would have different implications if they come at 
the same, lower, or higher spending levels than in 
the current (post—April 2017) system. 


Taxpayers fund New York City charter schools in 
several ways. The largest source of public funding— 
intended to cover operating costs—comes from 
a per-pupil allocation that is set by the state but 
paid by the school district through a pass-through 
fund. Charters can receive allocations for other 
expenses, though these costs tend to be minor. 
About two-thirds of charter schools also operate 
rent-free in a traditional public school building,”° 
while charters that do not operate in a traditional 
public school building often receive other in-kind 
support, such as food service and transportation. 


New York City charter schools receive, on average, 
less public support per pupil than do traditional 
public schools (IBO 2010, IBO 2017, Baker and 
Ferris 2011, Maloney and Wolf 2017). The differ- 
ence in total public funding between charter and 
traditional public schools in New York is substan- 
tial, depending on whether the school is located 
in a traditional public school building. The most 
recent estimate from the city’s Independent 
Budget Office (IBO 2017) found that, in 2016-17, 
the average per-pupil expenditure in New York’s 
traditional public schools was 5.7% higher than 
for charters in a traditional public school build- 
ing, 10.2% higher than for charters that received 
additional resources for their lease payments, and 
24.2% higher than for charters that received no 
capital spending supplement. 


“Fair student funding,” the funding formula for 
New York City’s traditional public schools, weights 
resources according to student characteristics, 
such as whether the student is eligible for free 
lunch and whether the student is classified as an 
English language learner (ELL). Baker and Ferris 
(2011) observe that proportionally fewer charter 
school students fall into each of these categories 
than do students in traditional public schools. 
Thus, if charter schools were funded according to 
the city’s funding formula for its traditional public 
schools, they would, on average, be expected to 
receive fewer dollars per pupil than the average 
city traditional public school. 


Still, the difference in the proportion of charter 
and traditional public school students classified as 


ELL has dropped from 14 percentage points at the 
time of the Baker and Ferris study, to only about 
7 percentage points today (New York City Charter 
School Center 2017). Thus, the cost-adjusted 
funding difference found by Baker and Ferris is 
likely—I am not aware of a recent update to this 
calculation—to be less important today, even as 
longitudinal results from matching studies suggest 
that average charter school effects have remained 
relatively consistent over time. 


Charter schools also receive resources from non- 
public sources, especially private foundations. 
Using data from audited annual financial reports 
from each charter school, Maloney and Wolf 
(2017) find that, on average, New York City charter 
schools received $349 per pupil from nonpublic en- 
tities, including $282 per pupil from private phil- 
anthropic sources. They also find that New York 
City’s traditional public schools received $841 per 
pupil from nonpublic sources. As such, philan- 
thropic funding does not appear to be a primary 
driver of the generally positive effects of charter 
schools relative to traditional public schools in 
New York. 


The widespread perception that charter schools 
receive substantial nonpublic resources to sup- 
plement their activities appears to be driven by 
outliers. Maloney and Wolf find that the vast ma- 
jority of philanthropic support for charters went 
to the top 25% of fund-raising schools. Using data 
from 2008 IRS 990 forms, Baker and Ferris (2011) 
similarly find that the amount of additional re- 
sources from private sources varies dramatically 
by charter school. For example, Harlem Village re- 
ceived more than $14,000 per pupil, while many 
other charter schools receive little, if any, philan- 
thropic support. 


Further, the empirical evidence offers little guide 
to whether variation in the amount of nonpublic 
resources available to a charter school has a mean- 
ingful impact on performance. For example, Baker 
and Ferris find no significant relationship between 
charter school spending (including public and 
private sources) and test scores—though this anal- 
ysis uses aggregate test scores adjusted for student 
characteristics as a measure of charter school 
quality, which is not a design that could plausibly 
lead to a causal estimate of charter school effects. 


Do Charter Schools 
Experience 
Disproportionate 
Student Attrition? 

Are Low-Performing 
Students Systematically 
Pushed Out of Charters? 


No and no. Students are less likely to exit charter 
schools than they are to exit traditional public schools, 
especially after accounting for demographic differences. 


Low-performing students are less likely to exit charter 
schools than they are to exit traditional public schools. 


There is a common claim, supported by numer- 
ous anecdotes reported in the media, that a main 
reason for the higher test scores in charter schools 
comes from their systematic removal of their most 
difficult to educate students. It is certainly plausi- 
ble that there have been cases when charter schools 
(as well as traditional public schools) have not ap- 
propriately handled situations with individual stu- 
dents. However, the academic research literature 
strongly suggests that charter schools have not sys- 
tematically manipulated their enrollments in this 
manner. 


Overall, charter school students are as likely, or less 
likely, to leave their school for another school in the 
city—or to leave the school district entirely—as are 
students in traditional public schools. IBO (2015) 
compares the mobility patterns of students who, in 
the fall of 2008, enrolled in a charter school or in 
the nearest traditional public school. It finds that, 
three years after entering school, about 64% of stu- 
dents who originally enrolled in a charter remained 
in that school four years later, compared with 56% 
of students who originally enrolled in a traditional 
public school. Using student-level data from grades 
3-8 from 2007 to 2012, Winters, Clayton, and Car- 
penter (2017) find that the overall yearly attrition 
between the sectors was very similar but that at- 
trition from charters was significantly lower than 
attrition from traditional public schools after ac- 
counting for student characteristics. 


Winters, Clayton, and Carpenter evaluate whether 
charter schools systematically remove students 
who are not performing well on standardized tests. 
They use student-level data for 2007-12 to compare 
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FIGURE 4 


Student Demographics, 2015-16 
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the attrition patterns of students in New York City 
charter and traditional public schools overall and 
by test scores. They find that low-performing stu- 
dents are much more likely to exit their school 
than are higher-performing students, regardless 
of the sector. The relationship between students’ 
test scores and the probability that they left their 
school did not vary depending on whether students 


30% 40% 50% 60% 70% 


attended a charter or a traditional public school. 
Because Winters, Clayton, and Carpenter find that 
attending a charter school reduced the likelihood 
that a student exited his or her school, they ulti- 
mately find that low-performing students in New 
York City are significantly less likely to exit charter 
schools than traditional public schools. 


How Do the 
Characteristics of 
Students Attending 
Charter Schools Differ 
from Students Attending 
Traditional Public 
Schools? 


Compared with those in traditional public schools, the 
average student attending a charter school is more likely 
to be black and have a lower socioeconomic status, but 


is less likely to be Hispanic, have a disability, or be an 
English language learner. These enrollment differences 
are substantial but appear to be gradually narrowing. 


Figure 4 illustrates the proportion of students in each 
sector in a variety of categories, as of 2015-16. A sub- 
stantially higher proportion of charter school students 
are black, and substantially fewer are white or Asian 
relative to the city’s traditional public schools. In ad- 
dition, the proportion of students with disabilities 
and those who are ELL is higher in traditional public 
schools than in charter schools. 


UFT (2010) and Baker and Ferris (2011) observe that 
the proportion of students in charter and tradition- 
al public schools who are eligible for free lunch (a 
measure of extreme poverty)—rather than simply re- 
duced-priced lunch—is substantially larger in tradi- 
tional public schools than in charter schools. According 
to the New York City Department of Education (DOE), 
the proportion of students eligible for free lunch in 
charter schools is undercounted.” Instead, Figure 4 
uses an “economic need index,” a measure of socioeco- 
nomic status that DOE uses to find peer schools with 
similar student characteristics for its accountability 
system (a higher value on the index represents a more 
impoverished student population). According to the 
index, students who attend charter schools are more 
disadvantaged, on average, than those who attend tra- 
ditional public schools. 


Figure 5 illustrates some changes in the demographic 
characteristics of New York City charter schools since 
2011-12. In recent years, it appears that the population 
of students enrolled in charters has become less black, 
more Hispanic, and with more ELL. Though the 
demographic enrollment gaps between charters and 
traditional public schools (Figure 4) are significant, 
current enrollment trends appear likely to reduce them. 


What Explains the 
Difference in the 
Characteristics of 
Charter and Traditional 
Public School Students? 


A smaller proportion of charter school students have 
disabilities or are ELL mainly because such students 
are less likely to apply to attend a charter school. 


Other hypothetical factors—such as skewed lotteries, 
differences in enrolling after winning a lottery, attrition 
patterns, and changes in classification status—have a 
limited impact, if any. 


The proportion of students in charter and traditional 
public schools who have a particular “classification,” 
such as a disability or ELL status, can be calculated by 
dividing the number of students with the classification 
by the total number of students in the sector. Any dif- 
ference between the two sectors in the proportion of 
students with a particular characteristic must there- 
fore be driven by a factor that contributes to the nu- 
merator and/or denominator of that calculation. Five 
factors are relevant: 


Such students could be less likely to entera 
charter school because... 


i. They are less likely to apply. 


2. After applying, they are less likely to win the lottery 
(i.e., the lottery is rigged). 


After winning the lottery... 
3. They are less likely to enroll. 
After entering the school... 


4. They are more (or less) likely to experience a 
change in their classification status. 


5. They are more (or less) likely to leave 
charter schools. 


Start with the factors that do not appear to meaningful- 
ly affect enrollment gaps. Are lotteries rigged (factor 2, 
above)? The RFT studies demonstrate that there is no 
significant relationship between the characteristics of 
students participating in the lottery and the character- 
istics of the students who were randomly offered a seat. 
This suggests that the lottery process is fair and does 
not influence enrollment gaps between sectors. 


13 


14 


New York City’s Charter Schools 


Are students with particular characteristics who win 
the lottery less likely to enroll (factor 3)? Charter 
schools, for instance, could inappropriately encourage 
students with particular characteristics to decline their 
acceptance offers. Of the RFT studies, only Hoxby and 
Murarka (2009) fully compare the characteristics of 
students offered a charter seat with those of students 
who actually enroll. They find no significant difference 
between accepted applicants and students who enroll— 
for gender, race/ethnicity, age at time of application, 
eligibility for free or reduced-price lunch, ELL status, 
or entering math or reading scores. They do find that 
students with disabilities are significantly less likely to 
accept a charter school enrollment offer, though the 
size of the difference is not substantial (7.61% of lottery 
winners were in special education, compared with the 
7.03% of lottery winners who enrolled). 


Unterman (2017) does not present a full set of these 
descriptive statistics in her RFT analysis of Success 
Academy schools. She does note that 51% and 43% of 
lottery winners were Hispanic or black, respectively; 
and of students who actually enrolled, 45% were His- 
panic and 40% were black. Unfortunately, she does 
not compare the difference in the proportion of lottery 
winners with ELL status with those who actually enroll. 


Are students with particular classifications more likely 
to leave charter schools (factor 5)? IBO (2014) and 
IBO (2015) find higher rates of attrition in traditional 
public schools than in charter schools when students 
were compared by gender, race/ethnicity, eligibility for 
free or reduced-priced lunch, ELL status, and disabil- 
ity status.?3 Winters (2013) and Winters (2014a) simi- 
larly find that students with disabilities or ELL status 
were less likely to leave their school if they originally 
enrolled in a charter than if they originally enrolled in 
a traditional public school. In other words, different 
rates of student attrition are an unlikely cause of en- 
rollment differences between the sectors. 


Are students with particular classifications in char- 
ters more likely to experience a change in their clas- 
sification status (factor 4)? Winters (2013) finds that, 
overall, charters and traditional public schools were 
equally likely to newly classify a student as having a 
disability. He also finds that charters were significantly 
more likely to declassify a student out of special edu- 
cation in early elementary grades—which partly drove 
the widening gap between the two sectors in the pro- 
portion of students enrolled in special education in 
those same grades. 


Winters (2014a) finds that the proportion of ELL stu- 
dents in 2010 who no longer had the ELL classifica- 
tion in 2011 was significantly higher in charter schools 


than in traditional public schools—a result likely influ- 
enced by the fact that the average student classified as 
ELL in a charter was more advanced in English than 
the average ELL student in a traditional public school. 
Winters also finds that because the initial proportion 
of charter school students with ELL status was much 
smaller than the proportion in traditional public 
schools, the ultimate impact of declassification was 
to reduce the special-education gap as students pro- 
gressed through school.** He finds that differences in 
declassification across sectors were largely responsi- 
ble for the narrowing of the ELL gap as students pro- 
gressed through grades. 


The research evaluating student enrollment data sug- 
gests that differences in the characteristics of charter 
and traditional public school students are not substan- 
tially influenced by factors that occur once students 
enroll in school. If factors 2—5 do not explain the dif- 
ference in the characteristics of students in charter and 
traditional public schools, factor 1—that some types 
of students are less likely to apply to charters—must 
explain it. And it does. 


All of the RFT studies find that students who applied 
to New York City charter schools were far more likely 
to be black—and less likely to be Hispanic or ELL—and 
have lower standardized test scores at entry than were 
students entering traditional public schools. There 
does appear to be considerable variation in special-ed- 
ucation applications. Hoxby and Murarka (2009), for 
instance, find that charter applicants were significantly 
less likely to be in special education than were other 
students in the city. Yet Dobbie and Fryer (2013) find 
no meaningful difference between the special-educa- 
tion status of applicants and other students in the city. 
Corcoran and Cordes (2015) even find that applicants 
to Democracy Prep charter schools were more likely to 
be in special education than was the average student in 
the city. 


Enrollment data on the universe of city schools also 
suggest that the application stage is the source of the dif- 
ference in the proportion of students classified as ELL 
or having a disability. Winters (2013) and IBO (2015) 
find large differences in the proportion of students in 
special education across the sectors at the point when 
students enter kindergarten. Winters (2014a) and IBO 
(2014) find large differences across the sectors in the 
proportion of new kindergarten entrants with ELL 
status. If charter lotteries are fair and certain types of 
students are not more likely to enroll after winning a 
lottery seat, the relatively fewer number of disabled and 
ELL students who enter charter schools is a product of 
relatively fewer applications by such students. 


Has Charter School 
Expansion Harmed the 
Performance of Students 
Attending Traditional 
Public Schools? 


No. Competition from charter schools has either 

no effect, or a positive effect, on the performance 

of students in nearby traditional public schools. 
Colocation with charter schools has either no effect, 
or a positive effect, on student performance in 
traditional public schools. When traditional public 
school performance improves, thanks to competition 


from nearby charter schools, the improvement may 
result partly from the increase in spending, per pupil, 
that occurs in the former when they lose students to 
charters (i.e., when a student leaves for a charter, the 
traditional public school retains some of the funds that 
were previously spent on the student, which leaves 
more money, per pupil, for the remaining students). 


An expansive charter sector has the potential to affect 
not only its own students but also those who remain 
in the traditional public school sector. Some suspect 
that competition from charter schools incentivizes tra- 
ditional public schools to improve in order to retain 
students. Others argue that charter schools harm tra- 
ditional public schools by siphoning away the most 
talented students and the education dollars that come 
with them. 


Numerous studies evaluate the effects of expanding 
charter sectors on student performance in tradition- 
al public schools. But only three such studies exist for 
New York City. These studies use a differences-in-dif- 
ferences design, which evaluates whether there was a 
change in the trajectory of student performance in a 
traditional public school that coincides with changes in 
the competition that it faced from charter schools. 


There are various ways to measure such competition. 
Cordes (2017) uses a geographic measure—specifically, 
whether any charter school operated within one mile 
of a traditional public school in a given year. Winters 
(2012) measures competition as the proportion of 
a traditional public school’s students who left for a 
charter school at the end of the previous year. Winters 
also examines a wider number of grades, while Cordes 
examines a longer period, though both conclude their 
evaluations between 2009 and 2011. 


Winters finds that increased competition from charter 
schools had no significant effect on student performance 
in traditional public schools. The estimated impacts 


were near zero and were precise. Cordes finds a small, 
statistically significant, increase in student performance 
in math and ELA. The positive effect increased as the 
traditional public school’s distance to the nearest charter 
school decreased. 


In the current environment of far more charter schools, 
their effect on traditional public schools may be differ- 
ent from that found in these studies. Still, the empirical 
research has found no evidence of charter schools nega- 
tively affecting the quality of traditional public schools— 
only evidence of a positive effect or no effect. 


If charter schools have a positive impact on the perfor- 
mance of traditional public schools, consider some of 
the implications. Figures 1-2 suggest that the effect of 
attending a New York City charter school on student 
performance has remained relatively stable over time. 
These estimates compare the outcomes of charter school 
students with those in traditional public schools. Thus, 
as traditional public schools improve, the effectiveness 
of charter schools must similarly improve for the impact 
of attending a charter school instead of a traditional 
public school to remain constant over time. 


In addition to competing for students and education 
dollars, New York charter schools could affect the per- 
formance of the city’s traditional public schools though 
colocation (sharing the same school building). About 
two-thirds of the city’s charters colocate with at least 
one traditional public school.”> Two studies estimate the 
impact of colocation with a charter school on the per- 
formance of students attending the colocated tradition- 
al public school. Both utilize a differences-in-differences 
design, which tracks student performance in a school 
before and after a charter is colocated. 


Winters (2014) finds that colocation with a charter 
school had no significant effect on the math or ELA test 
scores of students attending the colocated traditional 
public school. Cordes (2017) finds statistically signifi- 
cant test-score gains for students in the colocated tradi- 
tional public school—indeed, the gains were even larger 
than those experienced by students attending charters 
(i.e., the matching-based estimates described above). 


Cordes digs further in an effort to pinpoint the factors 
that caused the gains. She finds that competition from 
charters leads to a significant reduction in traditional 
public school enrollment, but such competition does 
not have a large effect on the characteristics of students. 
She finds that the loss of students to charters leads to a 
significant increase in per-pupil expenditures in tradi- 
tional public schools (for reasons explained above), even 
as overall spending declines (at least in the short run). 
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What Are Areas for 
Future Research? 


Student performance on standardized tests has been the 
overwhelming focus of the aforementioned research.” 
Unfortunately, there are no causal estimates of the 
effect of New York City charters on, say, high school and 
college graduation rates or on other life outcomes, such 
as earnings, incarceration, and teen pregnancy. 


Enrollment “backfill” (replacing students in charters 
when they leave for other schools) deserves more re- 
search, too. New York City charters are not required to 
backfill, and many do not: they worry that backfilling 
would change their school’s carefully cultivated student 
culture and thus, harm its effectiveness. If this fear is 
justified, it has important implications for charter school 
policy and for interpreting charter school effects. 


Other areas for future empirical research include mea- 
suring the effect of the charter sector on New York City’s 
teaching workforce, as well as on the (often poor) neigh- 
borhoods where charters are most numerous. 


Appendix: 


Conclusion 


The empirical research evaluating New York City’s 
charter school sector is not perfect. But it is not ambig- 
uous, either: charter schools have had positive effects 
on the students whom they educate, and they have had 
a positive or no effect on the quality of the city’s tradi- 
tional public schools. 


Still, further research on the New York City charter 
school sector is needed. Much of the existing evidence 
should be updated to ensure that the effects found in 
earlier studies have remained, even as the sector has 
expanded rapidly. More research is needed to un- 
derstand the exact mechanisms that produce these 
(largely) positive effects, the broader impact that char- 
ters have had on students and the city, and whether the 
lessons that we have learned about the charter sector 
might improve New York’s traditional public schools. 


Research on New York City Charter Schools 


Effect of Attending a New York City Charter School 
Caroline M. Hoxby and Sonali Murarka, “Charter Schools in New York City: Who Enrolls and How They Affect Their Students’ Achievement,” NBER Working 


Paper No. 14852, Apr. 2009. Hereinafter: Hoxby and Murarka (2009) 


Sean Reardon, “Review of ‘How New York City’s Charter Schools Affect Achievement,’ ” Education and the Public Interest Center & Education Policy Re- 


search Unit, Nov. 12, 2009. 


Will Dobbie and Roland G. Fryer Jr., “Are High-Quality Schools Enough to Increase Achievement Among the Poor? Evidence from the Harlem Children’s 
Zone,” American Economic Journal: Applied Economics 3, no. 3 (July 2011): 158-87. Dobbie and Fryer (2011) 


Center for Research on Educational Outcomes at Stanford University, “Charter School Performance in New York City,” Feb. 20, 2013. CREDO (2013) 
Will Dobbie and Roland G. Fryer uJr., “Getting Beneath the Veil of Effective Schools: Evidence from New York City,” American Economic Journal: Applied 


Economics 5, no. 4 (Oct. 2013): 28-60. Dobbie and Fryer (2013) 


Sean P. Corcoran and Sarah A. Cordes, “The Continuing Impact of Democracy Prep Public Schools: Preliminary Report,” New York University, July 2015. 


Corcoran and Cordes (2015) 


Rebecca Unterman, “An Early Look at the Effects of Success Academy Charter Schools,” MDRC, Aug. 2017. Unterman (2017) 
Center for Research on Educational Outcomes at Stanford University, “Charter School Performance in New York City,” Oct. 2017. CREDO (2017) 


Effect of New York City Charter Schools on Student Performance in Traditional Public Schools 


Marcus A. Winters, “Measuring the Effect of Charter Schools on Public School Student Achievement in an Urban Environment: Evidence from New York City,” 


Economics of Education Review 31, no. 2 (Apr. 2012): 293-301. Winters (2012) 


Marcus A. Winters, “The Effect of Co-Locations on Student Achievement in NYC Public Schools,” Manhattan Institute, Feb. 25, 2014. Winters (2014) 
Sarah A. Cordes, “In Pursuit of the Common Good: The Spillover Effects of Charter Schools on Public School Students in New York City,” Education Finance 


and Policy, July 2017. Cordes (2017) 


Issues Related to Charter School Enrollments 
Hoxby and Murarka (2009). 


Sean Reardon, “Review of ‘How New York City’s Charter Schools Affect Achievement,’ ” Education and the Public Interest Center & Education Policy Re- 
search Unit, Nov. 12, 2009. 


United Federation of Teachers, “Separate and Unequal: The Failure of New York City Charter Schools to Serve the City’s Neediest Students,” Jan. 2010. UFT (2010) 


Independent Budget Office of the City of New York, “Comparing the Level of Public Support: Charter Schools Versus Traditional Public Schools, Feb. 2010. 
IBO (2010) 


Dobbie and Fryer (2011). 


Bruce D. Baker and Richard Ferris, Adding Up the Spending: Fiscal Disparities and Philanthropy Among New York City Charter Schools (Boulder, CO: Na- 
tional Education Policy Center, 2011). Baker and Ferris (2011) 


Jack Buckley and Carolyn Sattin-Bajaj, “Are ELL Students Underrepresented in Charter Schools? Demographic Trends in New York City, 2006-2008,” Jour- 
nal of School Choice 5 (2011): 40-65. 


Winters (2012). 


Robin Lake, Betheny Gross, and Patrick Denice, “New York State Special Education Enrollment Analysis,” Center for Reinventing Public Education, Nov. 
2012. 


Center for Research on Educational Outcomes at Stanford University, “Charter School Performance in New York City,” Feb. 20, 2013. CREDO (2013) 


Marcus A.Winters, “Why the Gap? Special Education and New York City Charter Schools,” Center for Reinventing Public Education & Manhattan Institute, 
Sept. 30, 2013. Winters (2013) 


Dobbie and Fryer (2013). 

Stephen Eide, “Should Charter Schools Pay Rent? Implications for Staffing and Growth,” Manhattan Institute, Feb. 6, 2014. Eide (2014) 

Winters (2014). 

John Kucsera with Gary Orfield, “New York State’s Extreme School Segregation,” Civil Rights Project, Mar. 2014. 

Marcus A. Winters, “Why the Gap? English Language Learners and New York City Charter Schools,” Manhattan Institute, Oct. 6, 2014. Winters (201 4a) 


New York City Charter School Center, “Stil! Building Inequality: How New York’s Charter Public School Students Continue to Bear the Cost of School Facili- 
ties,” Nov. 2014. 


Independent Budget Office of the City of New York, “Comparing Student Attrition Rates at Charter Schools and Nearby Traditional Public Schools,” Jan. 
2015. IBO (2015) 


Susan Aud Pendergrass and Nora Kern, “Waiting for Their Chance: A Closer Look at Wait Lists in Urban Public Charter Schools,” National Alliance for Public 
Charter Schools, May 4, 2015 


Corcoran and Cordes (2015). 

Center for Research on Educational Outcomes at Stanford University, “Urban Charter School Study: Report on 41 Regions,” 2015. 
Democracy Builders, “No Seat Left Behind: The Unfilled Potential of Empty Seats in Charter Schools,” 2015. 

New York City Charter School Center, “New York City Charter Schools: 2016-17 Enrollment Lottery Estimates,” June 2016. 

Halley Potter, “Diversity in New York City’s Universal Pre-K Classrooms,” Century Foundation, Sept. 20, 2016. 

New York City Charter School Center, “Neighborhoods Primed for Charter School Growth,” 2016. 


Independent Budget Office of the City of New York, “Staff Costs and Pass-Through Payments to Charter and Nonpublic Schools Propel Education Depart- 
ment Spending,” Feb. 2017. 


Marcus A. Winters, Grant Clayton, and Dick M. Carpenter Il, “Are Low-Performing Students More Likely to Exit Charter Schools? Evidence from New York 
City and Denver, Colorado,” Economics of Education Review 56 (Feb. 2017): 110-17. 


Independent Budget Office of the City of New York, “With State Formula for Charter School Funding Likely to Change, City Costs to Grow More than 
Budgeted,” Mar. 2017. IBO (2017) 


Cordes (2017). 

Larry D. Maloney and Patrick J. Wolf, “Charter School Funding: Inequity in New York City,” University of Arkansas, Aug. 2017. Maloney and Wolf (2017) 
Unterman (2017). 

CREDO (2017). 

Clive Belfield, NEPC Review: Charter School Funding: Inequity in New York City (Boulder, CO: National Education Policy Center, 2017). Belfield (2017) 
New York City Charter School Center, “English Language Learners and NYC Charter Schools,” 2017. 

New York City Charter School Center, “New York City Charter Schools Are in Demand...and Growing,” 2017. 

New York City Charter School Center, “Students with Disabilities and NYC Charter Schools,” 2017. 


Issues Related to Charter School Finance 


Independent Budget Office of the City of New York, “Comparing the Level of Public Support: Charter Schools Versus Traditional Public Schools,” Feb. 2010. 
Baker and Ferris (2011). 


Joshua D. Angrist, Parag A. Pathak, and Christopher R. Walters, “Explaining Charter School Effectiveness,” American Economic Journal: Applied Economics 
5, no. 4 (Oct. 2018): 1-27. 


Eide (2014). 

New York City Charter School Center, “Stil! Building Inequality.” 

Independent Budget Office of the City of New York, “Staff Costs and Pass-Through Payments.” 
Independent Budget Office of the City of New York, “City Costs to Grow More than Budgeted.” 
Cordes (2017). 

Maloney and Wolf (2017). 


MI New York City’s Charter Schools 


Endnotes 


1 |have, to my knowledge, included all relevant direct analyses of New York City’s charter school sector in this review. If | have missed any relevant 
research, that is my regrettable error. | can say with confidence, however, that whatever research | may have missed has not played a meaningful role in 
New York’s charter school conversation—it is neither cited in the academic literature nor has it influenced the policy conversation. 


2 For instance, a 2003 report by the Institute for Education and Social Policy at New York University considered whether a charter school authorizer should 
use its authority to close ineffective charter schools. The issue of accountability for the charter sector remains important. However, the context for that 
conversation has changed dramatically since 2003. Thus, | do not include that paper in this review. 


3 See, e.g., the United Federation of Teachers, New York City Charter School Center, Independent Budget Office, and the National Education 
Policy Center. 


4 The seven studies are: Caroline M. Hoxby and Sonali Murarka, “Charter Schools in New York City: Who Enrolls and How They Affect Their Students’ 
Achievement,” NBER Working Paper No. 14852, Apr. 2009; Will Dobbie and Roland G. Fryer Jr., “Are High-Quality Schools Enough to Increase 
Achievement Among the Poor? Evidence from the Harlem Children’s Zone,” American Economic Journal: Applied Economics 3, no. 3 (July 2011): 158- 
87; Center for Research on Educational Outcomes at Stanford University, “Charter School Performance in New York City,” Feb. 20, 2013; Will Dobbie 
and Roland G. Fryer Jr., “Getting Beneath the Veil of Effective Schools: Evidence from New York City,” American Economic Journal: Applied Economics 
5, no. 4 (Oct. 2013): 28-60; Sean P. Corcoran and Sarah A. Cordes, “The Continuing Impact of Democracy Prep Public Schools: Preliminary Report,” 

New York University, July 2015; Rebecca Unterman, “An Early Look at the Effects of Success Academy Charter Schools,” MDRC, Aug. 2017; and 

Center for Research on Educational Outcomes at Stanford University, “Charter School Performance in New York City,” Oct. 2017. 


5 Using simulations and data for Boston, Walters finds evidence of an inverse relationship between preference for charter schooling and the effect of 
attending a charter school—i.e., within the Boston context, it appears that the students least likely to apply to charter schools would benefit from them 
the most. See Christopher R. Walters, “The Demand for Effective Charter Schools,” NBER Working Paper No. 20640, Feb. 2017. 


8 Each of these studies reports several different estimates. | focus on treatment-on-the-treated estimates that evaluate the effect of attending a charter. 
When a paper examines multiple grades, | report a representative finding. | also limit my discussion to the overall effects of charter schools and not how 
effects vary by student or school characteristics. 


7 An estimate is said to be statistically significant if we are at least 95% confident that the effect, whether positive or negative, is not zero. Statistical 
significance takes into account both the magnitude of an estimate and the precision with which it is estimated. An estimate of small magnitude might 
be statistically significant if it is estimated precisely, while an estimate of large magnitude might be statistically insignificant if it is estimated imprecisely. 
The precision of the estimate is affected by the variability of the result, as well as by the number of observations included in the analysis. 


8 Each study reports a variety of estimates, to confirm the results’ robustness or to find subtle differences in effect. | focus on the estimate that best 
reflects the overall finding of the respective study. However, my choice of the primary estimate is not always the same as the author’s choice. For 
example, the CREDO reports highlight estimates that combine four years of observations together; yet for the purposes of this review, the most relevant 
estimates are for each year separately. 


3 Six of the seven studies report the effect on a per-year basis. Unterman (2017) reports the effect after a period of four years. | translated the latter 
result into a per-year estimate by dividing it by four. 


10 The Krueger estimates suggest that the effect of small class sizes was mostly attributable to the first year of the treatment. See Alan B. Krueger, 
“Experimental Estimates of Education Production Functions,” Quarterly Journal of Economics 115, no. 2 (May 1999): 497-532. CREDO (2013) and 
CREDO (2017) translate effects from standard deviation units into “days of additional learning.” According to this translation, an increase of 0.0017 
standard deviations represents the equivalent of an additional day of schooling. To find the estimated increase in days of schooling, one would multiply 
the estimate in standard deviation units by 0.0017. This type of translation has gained some popularity: it is somewhat reasonable, and it is easily 
understandable for the lay reader. | encourage the reader to perform this calculation if it helps to put the overall effect into context. Nonetheless, the 
“days of additional learning” translation is controversial and, at best, imperfect, so | have not presented the results in this way. 


1 Seen. 7 above. 
12 Seen. 8 above. 
13 See n. 9 above. 
14 Seen. 10 above. 


18 Likewise, estimates from Cohodes, Setren, and Walters find that the average effect of Boston’s charter school sector has not diminished—and has likely 
increased—even as it doubled its market share in that city. See Sarah Cohodes, Elizabeth Setren, and Christopher R. Walters, “Can Successful Schools 
Replicate? Scaling Up Boston’s Charter School Sector,” School Effectiveness & Inequality Initiative, Oct. 2016. 


16 Dobbie and Fryer (2011) find that when these activities were adopted in Houston’s traditional public school system, they led to similarly positive effects 
in student achievement. 


17 Angrist, Pathak, and Walters (2011) find that “no excuses” schools with similar characteristics are much more effective than other schools in Boston. 
18 It is not clear from the paper whether the per-pupil spending measure includes expenditures from public and private sources. 

19 Monica Disare, “After Heated Debate, New York Charter Schools Receive Boost; School Aid Increases by $1.1 Billion,” Chalkbeat, Apr. 10, 2017. 
New York City Charter School Center, “Facilities.” 


See New York City Department of Education, “Data About Schools.” For the economic need index, see New York City Department of Education, “2016- 
17 School Quality Reports.” The index is calculated as: (% temporary housing) + (% HRA-eligible x 0.5) + (% free-lunch-eligible x 0.5). HRA-eligible refers 
to students whose families receive certain types of public assistance, as identified by the New York City Human Resources Administration. 


“Approximately 25% of charter schools in NYC do not utilize NYC DOE School Food to provide meal services. The NYC DOE Office of School Food does 
not collect documentation on students’ eligibility for free or reduced price lunch from schools that do not utilize NYC DOE school food. As a result, the 
poverty figures may be understated for approximately 25% of charter schools.” See New York City Department of Education, “Demographic Snapshot.” 
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23 |BO (2014) reports that students with disabili 


ities were much more likely to exit charter schools than traditional public schools. This finding was the result 


of a data error and was corrected in IBO (2015). See Sarah Darville, “IBO Admits Charter School Special Ed Attrition Numbers Missed the Full Picture,” 


Chalkbeat, Feb. 19, 2014. 


24 As far as the gap narrowing, Winters (2014a 
with 100 students in traditional public schoo 


observes: “Consider a simple, stylized example, one with 100 students in the charter sector and another 
s. In charters, two of the 100 students (2%) are classified as ELL, compared with 20 of 100 students (20%) 


in traditional public schools. If, hypothetically, the next year charter schools declassified one of their ELL students, charters would have declassified 50% 


of their ELL population that year; if traditiona 


public schools declassified five of their ELL students, traditional public schools would have declassified 


25% of their ELL population, respectively. Charters would therefore have a larger rate of declassification. Yet in this example, the ELL gap has 
nonetheless closed from 18% (20%-2%) to 14% (15%-1%). The same holds true for the actual data discussed earlier: the impact of charter schools 
declassifying a larger share of their ELL students each year is to reduce the ELL gap over time.” 


25 New York City Charter School Center, “Facili 


ies 


26 Corcoran and Cordes (2015) measure the eff 


ect of attending a Democracy Prep school on student attendance, and Dobbie and Fryer (2013) examine 


attendance and on-time matriculation. These metrics are worth measuring but cannot be considered educational outcomes. 
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Abstract 


This report evaluates the current state of research on New York City charter 
schools. Overall, their effect on student performance is unambiguously 
positive. But the research is more dated and limited in scope than 
proponents and critics of charters appreciate. 


Key Findings 


iL. 


Students who attend a New York City charter instead of a traditional public 
school do much better on math tests and better, but by a smaller amount, 

on English language arts (ELA) tests. These positive effects appear to have 
remained similar over time, even as the number of charters and students has 
expanded. However, the studies that employ the strongest potential research 
design are dated or targeted only at a few highly effective schools that do not 
represent the full charter sector today. 


. New York City charter schools are not equally effective. About half 


appear to be more effective than the traditional public school that 
students would have otherwise attended. A small percentage of charters 
appear to have negative effects on student test scores. 


. Differences in resources do not explain differences in effectiveness 


between charter schools and traditional public schools or between 
charters in New York City. 


. There is no evidence that New York City charter schools systematically 


push out low-performing students. In fact, low-performing students 
are less likely to exit charter schools than they are to exit traditional 
public schools, especially after accounting for differences in their 
demographic characteristics. 


. Competition from New York City charter schools has either no effect, 


or a positive effect, on the performance of students in the nearby 
traditional public schools. 
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